Skip to content

fix(devkit): sort inputs by line_number in load_inputs_and_outputs#4171

Closed
devteamaegis wants to merge 1 commit into
microsoft:mainfrom
devteamaegis:fix/sort-inputs-by-line-number-2646
Closed

fix(devkit): sort inputs by line_number in load_inputs_and_outputs#4171
devteamaegis wants to merge 1 commit into
microsoft:mainfrom
devteamaegis:fix/sort-inputs-by-line-number-2646

Conversation

@devteamaegis

Copy link
Copy Markdown

Problem

PFClient.runs.get_details() returned dataframe rows in inconsistent order, causing input rows to be paired with wrong output rows.

Root cause in LocalStorageOperations.load_inputs_and_outputs():

_outputs_padding() already sorts the outputs DataFrame by line_number ascending (line 504), but the inputs DataFrame was left in whatever order the SDK inputs JSON file was written — which depends on async executor completion order.

get_details() then merges inputs and outputs positionally via .to_dict("list") + DataFrame(data), so mismatched orderings cause silently wrong row pairings:

Row inputs (unsorted) outputs (sorted) Correct pairing?
0 line 2 → "third query" line 0 → "ans0" ❌ mismatched
1 line 0 → "first query" line 1 → "ans1" ❌ mismatched
2 line 1 → "second query" line 2 → "ans2" ❌ mismatched

Fixes #2646.

Fix

Sort inputs by line_number ascending inside load_inputs_and_outputs(), immediately after outputs are sorted and indexed, so both DataFrames are in the same order when get_details() merges them:

if LINE_NUMBER in inputs.columns:
    inputs = inputs.sort_values(LINE_NUMBER, ascending=True).reset_index(drop=True)

Tests

Added test_load_inputs_and_outputs_sorts_inputs_by_line_number to tests/sdk_cli_test/unittests/test_local_storage_operations.py:

  • Writes inputs file with rows in out-of-order sequence [line 2, line 0, line 1]
  • Writes outputs file with rows in sorted sequence [line 0, line 1, line 2]
  • Verifies load_inputs_and_outputs() returns inputs sorted as ["first", "second", "third"]

The test fails on main and passes with this fix.

get_details() merges inputs and outputs positionally: it calls
.to_dict("list") on both DataFrames and then constructs a combined
DataFrame from the parallel lists.

_outputs_padding() already sorts outputs by line_number ascending, but
the inputs DataFrame was left in whatever order the JSON file was written
— which depends on async executor completion order.  If inputs arrived as
[line 2, line 0, line 1] and outputs were [line 0, line 1, line 2], the
merged DataFrame incorrectly paired:
  - "third query" → "ans0"  (wrong: ans0 belongs to line 0 = "first query")

The fix sorts inputs by line_number inside load_inputs_and_outputs()
before returning, immediately after outputs are sorted, so both
DataFrames are in the same ascending order when get_details() merges them.

Fixes microsoft#2646
@devteamaegis
devteamaegis requested a review from a team as a code owner May 28, 2026 16:51
@devteamaegis
devteamaegis force-pushed the fix/sort-inputs-by-line-number-2646 branch from 854f028 to d498d10 Compare June 11, 2026 19:45
@github-actions

Copy link
Copy Markdown

Hi, thank you for your interest in helping to improve the prompt flow experience and for your contribution. We've noticed that there hasn't been recent engagement on this pull request. If this is still an active work stream, please let us know by pushing some changes or leaving a comment.

@github-actions github-actions Bot added no-recent-activity There has been no recent activity on this issue/pull request and removed no-recent-activity There has been no recent activity on this issue/pull request labels Jun 25, 2026
@github-actions

Copy link
Copy Markdown

Hi, thank you for your interest in helping to improve the prompt flow experience and for your contribution. We've noticed that there hasn't been recent engagement on this pull request. If this is still an active work stream, please let us know by pushing some changes or leaving a comment.

@github-actions github-actions Bot added the no-recent-activity There has been no recent activity on this issue/pull request label Jul 11, 2026
@github-actions

Copy link
Copy Markdown

Hi, thank you for your contribution. Since there has not been recent engagement, we are going to close this out. Feel free to reopen if you'd like to continue working on these changes. Please be sure to remove the no-recent-activity label; otherwise, this is likely to be closed again with the next cleanup pass.

@github-actions github-actions Bot closed this Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external no-recent-activity There has been no recent activity on this issue/pull request promptflow-devkit

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Inconsistent order for dataframe rows when loading run details via SDK

1 participant